Computational Vision
U. Minn. Psy 5036

Again try your luck against the Ideal Observer for dot density discrimination, but with a 2AFC task and a summary measure of your efficiency relative to the ideal.

Set up the mini-experiment

Here we make two improvements to the yes/no experiment.

First we'll turn it into a two-alternative forced-choice experiment. You'll experience how this makes the task subjectively easier, and reduces the problem of criterion fluctuation. 

Second, we will calculate your and the ideal's proportion correct, turn these into d's, and from there calculate your statistical efficiency. In 1997, Barlow reported an efficiency near 50%. How good are you?

As above, let's define a Poisson distribution with a mean of mean, with a function to draw a sample from this distribution.

For most psychophysical experiments it is a good idea to give the observer some practice trials. For these,  let highmean = 300 for the "high" setting, and lowmean =200 for the "low" setting.  Set numtrials=10; This will give you easy trials to get the hang of it

For the actual measurements, you want to make the task hard enough so that mistakes are made (why?). So make the task harder by setting highmean = 220 for the "high" setting, and lowmean =200 for the "low" setting.  Set numtrials=100;

z[p_]:=Sqrt[2] InverseErf[1-2 p];
dprime[x_]:=N[-Sqrt[2] z[x]];
dotsize = 0.01;
numberofphotons[mean_] := RandomInteger[PoissonDistribution[mean]];
highmean = 220;lowmean =200;
data={{"Was I Correct?","Was Ideal Correct?"}};

numtrials=10;

blank=Graphics[{PointSize[dotsize],Black,Point/@{{},{}}},AspectRatio->1,Frame->False,FrameTicks->None,Background->GrayLevel[0.0],PlotRange->{{-0.2,1.2},{-0.2,1.2}}];
flash=blank;

CreateDocument[Dynamic[flash],ShowCellBracket->False,WindowSize->{300,300},WindowMargins->{{Automatic,0},{Automatic,0}},WindowElements->{},Background->Black,NotebookFileName->"Flash Display"];

twoflashes:=Module[{tempmean},
Table[whichflash=RandomInteger[{0,1}];If[whichflash==1,leftnumsample=numberofphotons[highmean],leftnumsample=numberofphotons[lowmean]];If[whichflash==0,rightnumsample=numberofphotons[highmean],rightnumsample=numberofphotons[lowmean]];

leftsample=Table[RandomReal[{0,1},2],{leftnumsample}];
flash=Graphics[{PointSize[dotsize],Red,Point/@leftsample},AspectRatio->1,Frame->False,FrameTicks->None,Background->GrayLevel[0.0],PlotRange->{{-0.2`,1.2`},{-0.2`,1.2`}}];
Pause[.25];flash=blank;Pause[.25];

rightsample=Table[RandomReal[{0,1},2],{rightnumsample}];
flash=Graphics[{PointSize[dotsize],Red,Point/@rightsample},AspectRatio->1,Frame->False,FrameTicks->None,Background->GrayLevel[0.`],PlotRange->{{-0.2`,1.2`},{-0.2`,1.2`}}];
Pause[.25];flash=blank;

myanswer=ChoiceDialog["Did the signal (high density) appear on",{"First?"->1,"Second?"->0},WindowSize->{300,80},WindowMargins->{{Automatic,0},{Automatic,330}}];

If[myanswer==whichflash,WasICorrect=1,WasICorrect=0];idealanswer=If[leftnumsample>rightnumsample,1,0];If[idealanswer==whichflash,WasIdealCorrect=1,WasIdealCorrect=0];data=Append[data,{WasICorrect,WasIdealCorrect}],{numtrials}];
]

Execute a trial

Now, randomly turn the switch to "high" or "low", draw a sample, and then input your response (1 for "high" and 0 for "low). Execute the next cell 100 times.

twoflashes

Display the data

data//MatrixForm

Analyze the data

Let's drop the table heading stored in row 1, and then transpose the matrix so that the columns become the rows:

data2=Transpose[Drop[data,1]]

Let's use a combination of Map[ ] and Count[ ] (used earlier to make histograms) to count up all occurrences of  an event type. So the total for myhits is:

myproportioncorrect=N[Map[Count[data2[[1]],#]&,{1}]/Dimensions[data2][[2]]][[1]];
idealproportioncorrect=N[Map[Count[data2[[2]],#]&,{1}]/Dimensions[data2][[2]]][[1]];

mydprime=dprime[myproportioncorrect];
idealdprime=dprime[idealproportioncorrect];
mystatisticalefficiency=Round[100*(mydprime/idealdprime)^2];

Print[Style[Grid[{{"my prop correct","ideal's prop correct","my d'","ideal's d'","my efficiency (%)"},{myproportioncorrect,idealproportioncorrect,mydprime,idealdprime,mystatisticalefficiency}},Frame->All],9]];

To get a reliable estimate, you need at least 100 or more trials.

Adaptive procedures for finding thresholds using 2AFC or yes/no

What can you do if the human psychophysical observer is making lots of mistakes or alternatively getting all the trials right? The above method is called the "method of constant stimuli", because (although the stimuli really aren't constant), the conditions (highmean and lowmean) are. Adaptive or tracking methods are more efficient. The idea is to have a computer program automatically hunt for that threshold (e.g. highmean is adjusted) so that the observer is getting a prescribed proportion correct (e.g. 75%). There have been a number of advances in the art of efficiently finding values of a signal which produce a certain percent correct in a psychophysical task such as 2AFC. For more on this, see the QUEST procedure of Watson and Pelli (1983) and the analyses of Treutwein (1993).

References

Barlow, H. B. (1962). A method of determining the overall quantum efficiency of visual discriminations. Journal of Physiology (London), 160, 155-168.
Barlow, H. B. (1977). Retinal and central factors in human vision limited by noise. In B. H. B., & F. P. (Ed.), Photoreception in Vertebrates Academic Press.
Green, D. M., & Swets, J. A. (1974). Signal Detection Theory and Psychophysics . Huntington, New York: Robert E. Krieger Publishing Company.
Treutwein, B. (1993). Adaptive psychophysical procedures: A review. Vision Research
Watson, A. B., & Pelli, D. G. (1983). QUEST: A Bayesian adaptive psychometric method. 33, 113-120.